
\documentclass{beamer}




\usetheme{Darmstadt}
%\usetheme{Singapore}
\usepackage[english]{babel}


\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{bm}
\newcommand{\vm}[1]{% Vector or matrix
\bm{\mathrm{#1}}} 

\usepackage{verbatim}
%\usepackage{color}
%\definecolor{MyDarkBlue}{rgb}{0,0.08,0.45}
%\definecolor{MyBrickRed}{rgb}{0.45,0,0.05}
%\usepackage[colorlinks=true, linkcolor=MyDarkBlue, citecolor=MyDarkBlue, urlcolor=MyBrickRed]{hyperref}
\usepackage{hyperref}





\setbeamercovered{transparent}


%
% The following info should normally be given in you main file:
%


\title{Introduction to OpenBUGS Tutorial}
\author{Kevin M. Esterling (UC -- Riverside)}
\institute{}
\date{}


\begin{document}


\frame{\titlepage}

\section{Introduction}
\subsection{}





\frame[label=introtobayes1]{
	\frametitle{What is Bayesian inference? }
You are probably already familiar with frequentist inference \ldots
\begin{itemize}
\item Null hypothesis test procedure can tell you, if the true parameter is zero, what is the probability of observing a point estimate result of a given magnitude?
\item Or that a confidence interval will cover the true parameter 95 times out of 100
\item Neither of these interpretations is easy to communicate
\end{itemize}
\begin{center}
%\hyperlink{ideosd.densities}{\beamergotobutton{Empirical Distribution of Disagreement}}
\end{center}}


\frame[label=introtobayes2]{
	\frametitle{What is Bayesian inference?}
\begin{exampleblock}{Bayesian Inference} Recover the \textit{posterior} distribution of estimated parameters, given the model and data.
\end{exampleblock}
\begin{figure}[htb]
	\begin{center}
		\includegraphics[scale=0.25]{posteriorplot.pdf}
	%\caption{\label{fig:1}}
	\end{center}
\end{figure}
The goal of this tutorial is to show you how to estimate posterior distributions computationally using OpenBUGS
\begin{center}
%\hyperlink{ideosd.densities}{\beamergotobutton{Empirical Distribution of Disagreement}}
\end{center}}

\subsection{}

\frame[label=mantra]{
	\frametitle{Doing Bayesian statistical analysis}
\begin{block}{General form of Bayes rule for statistical modeling:}
\begin{center}
 $p(\beta | y) = \frac{p(\beta)p(y| \beta)}{p(y)}$
\end{center}
\end{block}
In words, the posterior density (beliefs after seeing the data) is proportional to the prior density (beliefs before seeing the data) times the likelihood of observing the data given those prior beliefs, divided by a normalizing constant. \newline \newline 
We can drop the normalizing constant that makes the posterior a true probability density
\begin{exampleblock}{}
\begin{center}
$p(\beta | y) \propto p(\beta)p(y| \beta)$
\end{center}
\end{exampleblock}
}


\frame[label=introtobayes3]{
	\frametitle{Doing Bayesian statistical analysis}
\begin{block}{Bayes Rule:}
\begin{subequations}
\begin{align}
\mbox{Posterior Beliefs} & = \frac{\mbox{Prior Beliefs} \times \mbox{Data Likelihood}}{\mbox{Probabilty of the Data}} \\
\ \vspace{24pt} &  \propto  \mbox{Prior Beliefs} \times \mbox{Data Likelihood} 
\end{align}
\end{subequations} %\newline}
\end{block}}





\frame[label=introtobayes2]{
	\frametitle{}
\begin{figure}[htb]
	\begin{center}
		\includegraphics[scale=0.45]{priorposteriorplot.pdf}
	%\caption{\label{fig:1}}
	\end{center}
\end{figure}

\begin{center}
%\hyperlink{ideosd.densities}{\beamergotobutton{Empirical Distribution of Disagreement}}
\end{center}}



\frame[label=introtobayes3a]{
	\frametitle{Doing Bayesian statistical analysis in the real world}

Bayes rule can sometimes be solved analytically, but Bayesian computational methods allow you to solve arbitrary, complex models much more flexibly and/or computationally faster. The goal of this tutorial is to show you how to estimate posterior distributions computationally using OpenBUGS.

\begin{center}
%\hyperlink{ideosd.densities}{\beamergotobutton{Empirical Distribution of Disagreement}}
\end{center}}



\section{Implementation}
\subsection{Example}


\frame[label=regdata]{
	\frametitle{Simulated Data Example}
\begin{math}  
\begin{array}{lll} %block
\mbox{xpostO1}_{i} & \sim & \phi(\mu_{i}, \tau)  \\
\mu_{i} & = & \beta_0 + \beta_{1}Site_{2i} + \beta_{2}Site_{3i}
\end{array} %block 
\end{math} \\

\vspace{24pt} Where, \\
$\beta_0 = 1.793$ \\
$\beta_1 = -0.690$ \\
$\beta_2 = 0.352$ \\
$\tau = 0.379 $ is the mean square precision

%\vspace{0.5in} \hspace{1.1in} \onslide<7>{\large{\textbf{\color<7>{green}Let's run this model in \texttt{OpenBUGS} \ldots}}}
}



\frame[label=regmodel1]{
	\frametitle{OLS regression model}
\noindent \textbf{Likelihood:} \\
\begin{math}  
\left.
\begin{array}{l}
\begin{array}{lll} %block
\mbox{Y}_{i} & \sim & \phi(\mu_{i}, \tau)  \\
\mu_{i} & = & \beta_0 + \beta_{1}X_{1i} + \beta_{2}X_{2i}
\end{array} %block 
\end{array} 
\right\} \mbox{$1 \leq i \leq \mbox{n.obs}$}  \onslide<2->{\mbox{\color<2>{red}\hspace{12pt}IID assumption}}
\newline \newline \textbf{Priors:} \newline
\begin{array}{lll} %block
\beta_0 & \sim & \phi(0, 0.0001) \onslide<3->{\mbox{\color<3>{red}\hspace{24pt}Flat priors (``uninformative'')}}\\
\beta_1 & \sim & \phi(0, 0.0001)\\
\beta_2 & \sim & \phi(0, 0.0001)\\
\tau & \sim & U(0, 1000) \onslide<4->{\mbox{\color<4>{red}\hspace{24pt}Flat positive prior}}
\end{array} %block
\end{math} \\  
%\vspace{0.5in} \hspace{1.1in} \onslide<7>{\large{\textbf{\color<7>{green}Let's run this model in \texttt{OpenBUGS} \ldots}}}
}



\subsection{}

\frame[label=computational]{
	\frametitle{Computational Bayesian statistics: MCMC}
\begin{itemize}
\item ``Bayesian estimation using Gibbs sampling'' (\texttt{OpenBUGS}) uses \textit{simulation} to approximate the posterior distribution
\item You give the software the \textbf{model} (priors and likelihood only), \textbf{data} and \textbf{starting values}, and the software will draw a sequence of realizations from the posterior distribution to create an empirical approximation of the posterior parameter distribution 
\item Basic procedure to run the simulation:
\begin{itemize}
\item Start at an arbitrary set of initial values
\item Discard ``burn-in period'' draws
\item Save and analyze ``stationary period'' draws
\end{itemize}
\end{itemize}
\begin{center}
%\hyperlink{ideosd.densities}{\beamergotobutton{Empirical Distribution of Disagreement}}
\end{center}}


\frame[label=mcmctable]{
	\frametitle{Computational Bayesian statistics: MCMC}
\begin{table}[p]
%\begin{changemargin}{-0.5in}{0in} 
\caption{Simulated Posterior Distribution \label{t:1}}
\begin{center}
\begin{tabular}{lccc} 
& $\beta_0$	& $\beta_1$	& $\beta_2$ \\
\hline
\textbf{Burn-in Period}&&& \\	
$t_0$	&20&-200&12 \\
$t_1$ &17&-105&15 \\
$t_2$ 	&2&-2& 2\\
$t_3$ &0.9&1.5& 0.0\\
\ldots &&&	 \\
$t_{9997}$	&0.7&1.7& 0.87\\
$t_{9998}$	&0.8&1.8&0.89	 \\
$t_{9999}$	&0.6&1.7& 0.95\\
$t_{10000}$	&0.6&	2.0& 0.91 \\
\hline
\end{tabular}
\end{center}
\end{table}}

\frame[label=mcmctable1]{
	\frametitle{Computational Bayesian statistics: MCMC}
\begin{table}[p]
%\begin{changemargin}{-0.5in}{0in} 
\caption{Simulated Posterior Distribution \label{t:1}}
\begin{center}
\begin{tabular}{lccc} 
& $\beta_0$	& $\beta_1$	& $\beta_2$ \\
\hline
\textbf{Stationary Period} &&& \\
$t_{10001}$	&0.7&1.9& 0.89\\
$t_{10002}$	&0.6&1.5& 0.87\\
$t_{10003}$	&0.8&1.6& 0.89\\
$t_{10004}$	&0.5&1.8& 0.83\\
$t_{10005}$	&0.7&2.1& 0.99\\
\ldots &&& \\	
$t_{10997}$	&0.4&2.2& 0.87\\
$t_{10998}$	&0.7&1.7& 0.97\\
$t_{10999}$	&0.6&1.9&	0.99 \\
$t_{11000}$	&0.9&1.5& 1.02\\
\hline
\end{tabular}
\end{center} %\vspace{-8mm}
%\end{changemargin}
%\begin{changemargin}{-0.5in}{0in} 
%\end{changemargin}
\end{table}}


\frame[label=computational2]{
	\frametitle{Computational Bayesian statistics: MCMC (cont.)}
Result is a simulated posterior distribution: computational approximation of the posterior
\begin{itemize}
\item The vector of draws post-convergence for each parameter is the marginal posterior distribution 
\item Summarize (mean, SD, 95\% intervals) and plot densities
\item Trivial to create sampling distributions of functions of parameters ($\widehat{\frac{ln(\beta_0)}{1+sin(\beta_1)}}$)
%\item Natural way to impute missing data for correct standard errors
\end{itemize}
\begin{center}
%\hyperlink{ideosd.densities}{\beamergotobutton{Empirical Distribution of Disagreement}}
\end{center}}


\frame[label=MCMCprocedure]{
	\frametitle{Computational Bayesian statistics: MCMC (even still cont.)}
\begin{exampleblock}{MCMC procedure}
\begin{enumerate}
\item Specify model (likelihood and priors) with \texttt{OpenBUGS} code
\item Create files with data and initial values (using RStudio)
\item In OpenBUGS:
\begin{itemize}
\item Check model, load data, and compile model
\item Provide initial values for parameters
\item Run model for an initial ``burn-in'' period until MCMC converges on the posterior distribution
\item Save a sample of draws from the posterior for parameters of interest
\end{itemize}
\item Summarize marginal distributions, plots, statistical tests
\end{enumerate}
\end{exampleblock}
\begin{center}
%\hyperlink{ideosd.densities}{\beamergotobutton{Empirical Distribution of Disagreement}}
\end{center}}


\section{Practical advice}


\subsection{Common problems....}

\frame[label=advice]{
	\frametitle{Common problems and some advice}
\begin{itemize}
\item Always run multiple chains (usually three) in order to test convergence
%\begin{itemize}
%\item BGR diagnostic assesses within-to-between chain variance (assumes overdispersed/random initial values)
%\item Consider both mathematical and empirical identification (just because you can write it down does not mean you should estimate it)
%\item Best to start with simple model and build up complexity
%\end{itemize}
\item Assess burnin period, mixing carefully, use BGR diagnostic 
\item Be sure there are no missing data on RHS
\item Read the manual; and Gelman and Hill (2006) is a great resource for multilevel modeling
\item Learn scripting language 
\end{itemize}
\begin{center}
%\hyperlink{ideosd.densities}{\beamergotobutton{Empirical Distribution of Disagreement}}
\end{center}}


\subsection{R interface}

\frame[label=Rinterface]{
	\frametitle{Using \texttt{OpenBUGS} with \texttt{R}}
In practice, you want to store your data and analyze/graph results within \texttt{R} (or \texttt{Stata} or \texttt{SAS} etc.)
\begin{itemize}
\item Once you know how to use \texttt{OpenBUGS} you can read documentation to these \texttt{R} packages:
\begin{itemize}
\item \textbf{\texttt{R2OpenBUGS}}, \texttt{BRugs} = Interact with \texttt{OpenBUGS} within \texttt{R}
\item \textbf{\texttt{CODA}} = Suite of tools to assess convergence and describe results
\item \texttt{BRugs} installs/loads all three
\end{itemize}
\item Call \texttt{OpenBUGS} from \texttt{R} for automating Bayesian analysis
\end{itemize}
\begin{center}
%\hyperlink{ideosd.densities}{\beamergotobutton{Empirical Distribution of Disagreement}}
\end{center}}



\end{document}



